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ABSTRACT 

Findings from an interdisciplinary research project, 
Methods of Analyzing Samples of Elicited Discourse in English and 
Spanish for Determining Student Language Proficiency, are presented. 
The research project was designed to develop a unified framework for 
the analysis of audiotaped discourse samples elicited from 
Spanish/English bilingual students that is directly applicable to 
assessing oral language proficiency. The framework for analysis 
integrated and applied recent research in a variety of fields 
including sociolinguistics , language acquisition, developmental 
pragmatics, the ethnography of communication, and the philosophy 
language. The paper summarizes information about the linguistic 
features, interactional communicative strategies, and discourse 
structures used by kindergarten through grade 5 students in 
conversation with an adult interlocutor and when telling narratives 
from wordless books. Criteria distinguishing communicative strategies 
used by students displaying a range of proficiency from high to low 
are described in terms of multiple co-occuring variables across 
lexical, clause, prosodic, and discourse levels. As elicitors 
discourse styles strongly influenced the complexity and coherence of 
language produced by students, criteria were developed to describe 
and evaluate the quality of the elicitation process. Problems and 
recommendations in the application of a sociolinguistic discourse 
analysis approach 
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This paper summarizes findings from an interdisciplinary 
research project, Methods of Analyzing S amples of Elicited Discourse 
in Enqlish and Sp anish for Determining Student Lan guage Proficiency. 
The research project was designed to develop a unified framework 
for the analysis of audiotaped discourse samples elicited from 
Spanish/English bilingual students that is directly applicable to 
assessing oral lanquage proficiency. The framework for analysis 
Integrated and applied recent research in a variety of fields in- 
cluding sociolinguistics, language acquisition, developmental 
pragmatics, the ethnography of communication and the philosophy ot 
language The paper summarizes information about the linguistic 
features, interactional communicative strategies, and discourse 
structures used by kindergarten through grade five students in con- 
versation with an adult interlocutor and when telling narratives 
from wordless books. Criteria distinguishing communicative stratenies 
used by students displaying a range of proficiency from high to ow 
are described in terms of multiple co-occurmg variables across lexical, 
clLse, proscdic and discourse levels. As elicitors discourse ; styles 
strongly influenced the complexity and coherence of language produced 
by students, criteria was developed to describe and evaluate the 
quality of the elicitation process. Problems and recommendations in 
the application of a sociolinguistic discourse analysis approach to 
language proficiency assessment in school districts is discussed. 

The research project was the result of a collaborative effort 
between Helen Slaughter, Department of Legal and Research Services, 
Tucson Unified School District and Adrian Bennett who was n the 
Department of Anthropology, University of Kentucky during 1980-81 
and is now at Centro de Estudios Puertorriquenos in New York -City. 
Helen Slaughter assumed major responsibility for the final chapter 
(VII) 'on a Sociolinguistic Discourse Alternative for Language Pro- 
ficiency Assessment" which forms the basis of this report. Adrian Bennett 
wrote the initial framework for Analysis of Elicited Discourse Samples 
upon which the research was based. Analyses and transcripts ot 
students' Spanish discourse, based on applications of the framework, 
written by Olivia Arrieta, Otto Santa Ana-A and Betty Garcia, were 
instrumental in conducting the research project. 



A Sociolinguistic-Discourse Alternative 
for Language Proficiency Assessment 



This research has been a pioneering attempt to base language 
proficiency assessment in a theory of communication as human^ 
interaction. Language proficiency was defined as communicative 
competence and, following Hymes (1972), the data of communicative 
competence was viewed as the interaction of language and social 
settings. A distinctive feature of this approach was a 
delibera-te focus upon context, and the way students as well as 
adult examiners created the interactional and linguistic context, 
rather than presupposing context as a given. Tne following 
discussion will summarize some of the major points of the study 
and relate them to practical requirements of language proficiency 
assessment in. school districts. The advantages and disadvantages 
of a sociolinguistic approach to assessment will be detailed as 
well as recommendations for further research. 

Direct Measurement of Speaking and 
LTstening in A Naturalistic Context 

In this study an attempt was made to simulate, as nearly as 
possible, an assessment situation in which the student's communi- 
cative competencies could be evaluated on the basis of the way 
speaking and listening function in ordinary social contexts. 
Discourse was elicited from students in an oral language inter- 
view durinq which an adult examiner engaged the student in 
conversation and encouraged him/her to talk freely upon several 
topics mutually arrived at between the two during the course of 
the conversation. Conversational elicitation was followed 
by asking the student to tell a story from a wordless picture 
book after the student had been given the chance to look through 
the book to discover the story sequence. The elicitation 
procedure, conversation and story-telling task, was first 
attempted in the minority and/or home language, i.e., Spanish, 
followed by an elicitation in English. Different but similar 
wordless books (Mercer and Marianna Mayer), were used in the 
Spanish and English elici tations . 

These procedures, by placing the student into an interactive, 
speaker-listener role, constitute a direct measure of communi- 
cative proficiencies or competencies where the mul ti faceted , 
synchronous aspects of oral communication, (i.e., semantic, 
syntatic, lexical, prosodic and pragmatic), can be exhibited and 
analyzed. Wallat and Green (1980) describinq the sociolinguistic 
constructs underlying the study of communicative competence 
state that the various aspects of communication cannot be unde r- 
stood in isolation from one another because "these features are 
not separate cues to meaning; rather, they occur in varying 
combinations and provide a degree of redundancy for message 
interpretation . 11 



A student's' ability to interpret and provide various communi- 
cative cues co-ocurring within a context created during a 
conversational interview is a broader view of competency or 
proficiency than found in other, more indirect approaches to 
language assessment. Indirect measures of language out of 
context, such as sentence repetition tasks or multiple choice 
tests of grammatical correctness, may produce results that 
falsely suggest a student knows how and when to use a form s/he 
canrYot use or conversely that s/he cannot produce a structure 
when required to in other social contexts. 

Language acquisition researchers also have often tested the 
acquisition of complex structures' under experimental conditions 
that have placed children in an artificial situation where the 
normal contextual and paralinguistics cues accompanying speech 
have been removed. This has lead some researchers to question 
the validity and generalizability of knowledge of language 
acquisition based upon this so called scientific research 
tradition (Karmeloff-Smith , 1979). Furthermore, indirect 
measures of language competency assume a single standard 
"language, " rather than recognizing functionally oroficient 
dialectal variations, and assume a 'shared, but unspoken, back- 
ground understanding of the demands of the assessment task that 
may not be valid investing linguistic minority students. 
Comparative psychologists (Cole and Means, 1981) have shown there 
are narked cultural differences in subjects perceptions of 
psychological tasks but that performance differences seem to 
disappear when supporting contexts are provided. 

The elicitation of conversation and narrative discourse can 
be viewed as midway between an ethnoaraphic-sociolinauistic study of 
spontaneous speech produced in a variety of interactional 
situations and more highly structured and indirect measdres 
of specific linguistic abilities. Spontaneous speech in natural 
social contexts has often been used by soci ol inqui sts and other 
linguists to study language proficiencies and/or more broadly 
conceived communicative competencies of a small sample of children. 
Some measurement specialists in education have also considered 
spontaneous speech samples to be "the most suitable method for 
evaluating which morphological and syntactic rules and structures 
the child has learned readily in his language," (Wiig and Seme! , 
1980, p. 97) however, they also view the disadvantages of this 
method of basing even clinical judgments about students language 
proficiencies to be overwhelming in terms of size of languaqe 
sample needed, difficulty of collecting data, and complexity 
and time required in analysis. The use of an elicitation 
strategy to obtain audiotaped data upon which to directly base 
judgments of languaqe Proficiency was mentioned by Rosansky (1980) 
* in her suggestion that modifications of the Foreign Service 
Institute Oral Language Interview Test for determining language 
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proficiency might be useful in school district assessment programs. 
There is also a growing recognition among test developers that 
speaking and listening skills should be tested directly under 
performance conditions requiring students to demonstrate their 
skills (NWREL, 1980). 

The question then of'whether and how well this situation can 
be said to represent a naturalistic context for assessing 
communicative competency must be addressed. The analysis of 
conversational data from this study has suggested that several 
different types of "contexts" may result from the interaction of 
an adult and a student during an oral language interview. These 
'different contexts, i.e., examination, interview or conversation, 
appear to have a marked influence upon the level and quality of 
discourse elicited from students, especially that of younger 
and/or less proficient students. A conversational context 
where the student is encouraged to engage in mutually established 
topics of conversation with the adult, and where the adult builds , 
upon the students responses, appears optimal for eliciting 
discourse. Conversely, an examination sequence, described by 
Mehan (1979) as a repetitive initiation, response, evaluation (IRE) 
pattern, tended to result in more sparce discourse production, 
especially in the younger age ranges, e.g. five to seven years 
old. 

An adult-child interrogation pattern was also found to 
result in sparce child discourse samples by Sulzby (1981, p. 84) 
in a study of Kindergarten students from highly literate home # 
backgrounds. Some students, especially older students, elaborated 
upon topics after single or "first" questions, which indicated 
that an interview pattern of initial open-ervded question and 
follow-up question would suffice to elicit an adequate discourse 
sample in these cases. A few students, perhaps perceiving that, 
the purpose of the "game," was for them to talk in an elaborated 
way, responded with extended discourse to questions that would 
have been given yes/no answers by other students. In other words, 
some students needed little in the way of background decontex- 
tualization from the examiner in order to elaborate upon topics 
while others only gave elaborated responses or even clause responses 
after the examiner had interacted conversationally wfth them over 
a number of turns on one or several topics. In the latter cases, 
the examiner must use questions and other utterances such as 
comments on the topic, that are situated in the ongoing conver- 
sational discourse (Cook-Gumperz , 1 977 ) . One indicator of the 
'naturalness' of the assessment context is the use of situated 
elicitations similar to natural conversation rather than a 
standard set of interview questions in an assessment process. 
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The question of the relevance ■ of an individual adult-child 
interview context outside of the classroom to within classroom 
and other socially valued contexts is an important one and one that 
we intend to pursue in future research. Some classroom communi- 
cative competencies, such as getting a turn, are not measured in 
this one-to-one context while others, such as holding the floor 
and knowing what and how much to say to whom are interpreted 
differently in a one-to-one adult/child conversational context 
than in a classroom context. However, we suspect future research 
would show that the student's ability to acti'vely participate in 
and sustain a conversation with an adult teafcher-exami ner would 
be, directly related to his or her ability to communicate with 
teachers in classroom discourse. Also, the ability to use 
discourse strategies to build a narrative text appears to.be 
an important aspect of proficiency in language use as it is 
encountered in schools in developing literacy skills. In their 
discussion of communicative competence in the classroom, Wallat 
and Green (1980) stated, "We still have not decided. . . whether 
we can realistically continue to assume that an ideal instruc- 
tional situation and ideal speech situation exists." Certainly, 
there are few opportunities for students to demonstrate 
productive communicative competencies in some classrooms. 
Mace-Matluck et al . (1981) found that bilingual students' language 
in the classroom was sparce when compared to that of their play- 
ground and home interactions. The authors concluded that the 
students had few opportunities to engage in extended discourse 
within the bilingual classroom. Therefore, an individual assessment 
situation may provide a much greater opportunity for students to 
display an optimal level of competency than does the typical 
classroom setting. 

A more important issue is whether children are required to 
talk about topics such as helping at home or how to play a game 
in naturally occuring speech in contexts other than the assessment 
situation. What are the topics that are familiar to students and 
which would they be likely to initiate in naturally occurinq 
conversations with adults, both within and outside of classroom 
contexts? More research is needed in this area. m 

Assessing Communicative Competencies: Criteria for Evaluation 

This study and the methods of analysis recommended in it 
for assessment were based upon a definition of lanauaae proficiency 
as communicative competence, in a variety of interactional 
contexts. Judqments of proficiency were seen to be based upon judgments 
of opiounicative intent. The units of analysis used for 
evaluating communicative competency are of the utmost importance 
in conclusions reached about what proficiency entails and also 
about the performance level of individual students. Two distinct, 
but overlapping, units of analysis, (1) interaction and (2) oral 
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discourse, formed the basis of the methods of analyzing elicited 
discourse developed within the course of this study. Both of 
these foci for analyzing language proficiency are relatively 
unique in terms of prevailing assessment methods used in education. 

Language testing has generally been based upon methods that 
removed or standardize "ouV meaningful conversational interaction 
between an examiner and student. As one example, of which there 
are many, in the Oral Language Evaluation (OLE) test (Silvaroli, 
1975) students were asked to talkabout a picture and examiners 
are adjured not to provide any more cues to the student. The 
analysis following the OLE test was based entirely upon syntactic 
structure with the assumption presumably made that the inter- 
actional variables are unimportant since the situation has been 
•neutralized.' Other test developers have attempted to eliminate 
interactional variables in the belief that to include them would 
create a haphazard and 'uncontrolled' or 'unscientific' assessment 
context and also produce insuperable problems for analysis and 
evaluation (Wiig and Seme! , 1979). In our research we have taker, 
the position that in order to be valid, oral language assessment 
must take place in a meaningful communication context where 
normal communicative cues are provided and that the resulting 
interactional variables can be systematically analyzed and 
integrated into evaluations of student communicative competencies. 

Few language proficiency measures have focused upon discourse 
or the larger units of meaning beyond the sentence level. In this 
study we have developed methods for analyzing (1) interactive 
discourse that occurs between conversational participants where 
linguistic context, i.e., previous utterances, are important to 
interpretation, (2) extended or elaborated discourse on conver- 
sational topics and (3) narratives told about stories in wordless 
books. The ability to produce coherent, extended discourse is 
an important characteristic in distinguishing between students who 
are proficient in a language and those who are not. The methods 
for analyzing both interactional and di scourse" prof iciencies were 
discussed in the full report. Suggestions for the application 
of this research to language proficiency assessment are given 
below. 

Interactional Proficiencies . There were a number of inter- 
actional proficiencies exhibited by students during the conversation 
and narrative elicitation that contributed to the development, 
maintanence and direction of the interactional situation ^and 
information conveyed. These included the. often unnoticed elements 
of appropriate backchannel feedback to the speaker and the 
prosodic patterning of responses as well as the more obvious 
conversational clarification strategies to be detailed below. 
We paid particular attention to the conversational context as 
negotiated between student and adult interlocutor rather than 
simply focusing upon student responses to the examiner. A 
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theoretical discussion of a socially active notion of context is 
found in ''Context in Children's Speech' 1 by Jenny Cook-Gumperz 
and John J. Gumperz (1975). . In order to focus uDon the mutual . 
development of context between examiner and student, it became 
necessary to consider larger segments of discourse in making - 
judgments about proficiency. In other words, the focus must 
be upon discourse level units of analysis which may contain a 
number of turns between examiner and student rather than a focus 1 
upon clausal level units or one examiner/student response intervals. 

The list of interactional proficiencies given below were 
these, derived from tte data, jihat appeared to contribute to the^ 
conprehensibility of line conversation and that indicated student's 
growing awareness of his/her role as a conversational partner. 
We omit from this section the discussion those semantic/pragmatic 
elements such as background information that will be part of the 
discussion of discourse proficiencies. 

Many of these interactional proficiencies are context-bound 
in that they can only be expected to appear in some conversational 
contexts and not others. For instance, an important skill is the 
detection and negotiated repair of misunderstandings occuring 
during the course of a conversation since such misunderstandings 
are not uncommon in naturally occuring speech and if undetected 
tend to break down communication. However, in some conversations, 
including many in our sample, misunderstandings did not occur and 
therefore the skill of negotiating a misunderstanding could not 
be observed. Also if misunderstandings occurred constantly this 
might be a sign of lack of proficiency. We feel that in evaluation 
it is important to distinguish between interactional behaviors 
sustaining or improving communication and those impairing 
communication. Also we hypothesize that clusters containing 
more instances and varieties of interactional proficiencies will 
be more descriptive of proficient students than of limited or 
less proficient students. In other words, -a particular behavior 
may not occur but some cluster of interactional skills will be 
observed in- more proficient students. In our data we observed 
a trend of more instances and varieties of interactional 
proficiencies in older student^. 

Interactional proficiencies observable in audiotaped student 
discourse are as follows: 

Responses 

Responses to elicitations were semanti'cTvi'y^ 
and pragmatically appropriate. 

Clarifications were provided when student was asked 
question^* about what he/she was talking. 
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Prosody 

Prosodic features, i.e., pitch, rhythm, loudness, 
were synchronous with ongoing interactional context, 
(e.g. pauses between/within utterances were within 
range expected of competent speaker of the language). 

Used prosody for conveying 'relationships between different 
parts- of the discourse (e.g. linkinq, contrasting, high- 
lighting) . 
Monitoring 

Detected and corrected examiner misunderstandings. 

Asked the examiner for clarification of task.' 

Elicited an evaluation from examiner. 

Asked whether or not the examiner understood what s/he 
said. 

Commented upon or* made asides about what s/he was 
talking about. 

Qualified or self-corrected information previously 
given. . 

Monitored speech resulting in imoroved communication 
by repairing or editing utterances. 

• Negotiated/Constructed the Context 

Successfully shifted topic of conversation. 
Used strategy of digressing from topic. 
Initiated topic of conversation. 

Changed registers, as in switching from a conversational 
to a narrative style or reducing/increasing the formality 
of the interaction. - % 

Made conversational intentions explicit, such as relating 
what was said to knowledge/experiences assumed to be 
shared with listener or in the listener's experience. 

Within the body of this report we have emphasized the importance 
of the role taken by the adult in eliciting language from students 
for language proficiency assessment. Student language proficiency 
cannot be evaluated without also evaluating examiner competencies 
in interacting with students. The adult plays a dominant role in 
conveying the communicative intentions established during the 
assessment procedures. Linguistic minority students may not 
always share the same perception of communicative interactions 
with the examiner, even when they are both from the same ethnic 
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background. For instance, some Hispanic children may assume that 
standards of respect and politeness call for brief responses to 
questions while others may assume that more elaboration is expected. 
We recommend that examiners' be trained in strategies for 
providing more explicit cues regarding the social and linguistic 
context to students who may be unfamiliar with the 'educator* 
expectations of the assessment context. 

Since the influence of the examiner upon language samples 
elicited from students is of fundamental importance to subsequent 
evaluations, it is suggested that the quality of the examiner's 
performance be evaluated previous to evaluating a student's 
language proficiency. Table 1 shows a format' being considered 
by the school district based upon this research for inclusion as 
part of the scoring system for the Language Proficiency Measure. 

Discourse Proficiencies . The discourse features listed below 
are those which have been -found in this study to be central in 
judgments of the language proficiency or communicative competency 
of linguistic minority students. While the interactional context, 
and thus, the opportunity or probability of these features 
occurring may be different in the conversational and wordless 
book tasks, these features are important in the assessment of 
elicited discourse samples from both tasks. Expected differences 
in performance dje to task and age or acquisitional level will be 
discussed. The features are: 

1. Coherence/Comprehensibil ity of Utterances, 

2. Appropriateness and/or Negotiated Context of Utterances, 

3. Complementarity as a Conversational Partner and 

" Ability to Produce Extended or Elaborated Discouse, 

4. Effective Use of Prosody, 

5. Provision of* Adequate Background Information Prior 
to Point Making , 

6. Completness of Information, 4 

7. Richness or Complexity, 

8. Flexibility and Range of Communicative Competencies, 

9. Pointmaking and Highlighting, 

10. Summarizing/Synthesizing, and 

11. Verb Tenses in Narrative Discourse. 

While these features may overlap in varying degrees in specific 
"instances, each will be discussed separately. 

1 . Coherence/Comprehensibil ity of Utterances . The most 
important feature of students language is to determine whether anG 
how much meaning is communicated to the listener. A focus on 
comprehensibility, 'especially when assisted in this interpretation 
by persons familiar both with children's lanquage and with the 
communication styles of the ethnic minority community, provides 



TABLE 1 

VACltfATION OF ASSESSMENT PROCEDURE 
Tape No. Analyst 



Quality of Recording : easy to hear difficult to hear 

Quality of Examiner's Performance : 

1. Generally, the examiner elicited conversation from the 
student by which method? 

examination ^interview cooperative conversation 

* 2. Did the examiner allow the student time to respond to the 
topic(s)? 



often sometimes rarely 



3. Did the examiner's responses to the student indicate s/he 
understood what the student said? 

often sometimes rarely 

4. Did the examiner build on the student's utterances? 
■ often sometimes rarel y 

5.. Did the examiner abruptly change topics? 
often sometimes rarely 

6, Did the examiner elicit a narrative from the wordless 
book by saying, "Tell me a story . .."? 

yes h o 

7. Did the .examiner attempt at least one descriptive and 
one expfanation-type topic and also a narrative from the 
wordless book? 

yes no' 

If so, what was missing? \ 

Students Whose Discourse was Difficult to Elicit : 

8 If the student tended to respond with "close down'.' strate 
gies, e.g.', "I don't know, 'that's all," did the examiner 
repeat or 'redirect the questions? 

often sometimes rarely not applicable 

9. If the student was reluctant to talk, did the examiner 
attempt to elicit language by providing context or back- 
ground information to the student? _ ■ _ 
often : sometimes rarely not applicab.e 

10. In the beginning of the wordless book, did the examiner 
provide additional probes to clari'fy the task? 
^ yes no not applicable 

Summary of Quality of Examiner's Performance : 

* excellent • adequate j 0 inadequate 
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a basis for language proficiency assessment where dialectal 
^variations and communicative intent can be considered when making 
judgments about proficiency. .* The comprehensi bi.l ity or sense- 
making aspect of the students language is fundamental to further 
and more detailed descriptions of proficiency. It is the point 
of departure for discourse analysts. In interpreting the 
language of young bilingual children especially it is important 
that analysts be> knowl edgeabl e about the character^ sties of child 
language and "the cultural backqround of students. 

2 . Appropriateness and/or Negotiated Context of Utterances . 
Judgments of language proficiency will include judgments about 
whether or not the students utterances are appropriate in terms 

of semantic and pragmatic context Such features as whether 
the student stays on the same topic as the examiner 4 , selects an 
appropriate register in terms of formality, informal Tty dimensions 
or genre, and selects the 'right' sort of thing to talk about, 
will affect judgments of proficiency. Black's .(1979) scale for 
rating younq" chi 1 dren ' s communicative competencies in the classroom 
contained items regardinq appropriateness such as the child's 
ability to vary responses according to conversational topic. 

The range of utterances accepted as appropriate will be much 
broader in a socio 1 inguistio discourse approach to assessment 
than those accepted in other types of lanquage assessment, 
especially that of discrete item tests./ This is partly because 
the student has an opportunity to construct the context as well 
as respond to the adults utterances during the interaction. We 
recognize a variety of .ways conversations can be meaningful, 
including explicitly shifting or chanqing topics of conversation 
and we recognize that there are a wide range of forms narratives 
can take. ; ' - 

3. Complementarity as a Co nversational Partner -and .Abil ity 
to Produce Extended of Elaborated Discourse . The model suggested 
by this approach to language proficiency assessment is based on \ 
the recognition that competencies in* face-to-face communication 
are central to issues. of language proficiency assessment. In 
assessing proficiency»data must be obtained indicating that the 
student can effectively interact as a conversational partner, over 

a number of turns. (Indicators of proficiency .during conversational 
interaction were listed previously. J We have also indicated that 
the ability to talk at length which we have very minimally defined 
as turns of talk of 3 or more clauses, is essentia-1 to demonstrating 
proficiency. Indeed, the. .wordless book/task, was included in , the . 
assessment procedures partly because it provided a* device for 
encouraging extended talk. Recognizing that students might have 
varying skilTs in oroducinq extended talk due, to topic or genre 
differences we have suggested that examiners attempt to elicit 



elaborated responses about (1) a process of doing?or making 
something, (2) narratives of personal or vicarious, i.e., TV, 
experiences, and (3) narratives from a wordless book. We also 
suggest that because the linguistic context, i.e., the influence 
of preceding- examiner utterances on the content and style of 
students discourse; is different in interactive and extended 
discourse,, that these differences be considered in evaluation. 

In summary, in analyzing student discourse we determine 
whether the students can effectively take the role of a conver- 
sational partner and whether or not the student elaborates on 
topics or builds a narrative text. 

4. Effective Use of Prosody . The importance of effective 
prosodic patterning has already been mentioned in the previous 
section as important in terms of the student's proficiency as an 
interactant with others. Effective communicatipn involves the 
ability to manipulate and monitor multiple levels of co-occurinq 
variables in a stream of discourse which has a melodic and 
rhythmic, as well as a grammatical and lexical organization 
(Bennett, 1980). Prosodic features are not a separate feature, 
but involve an intricate interplay with cues from semantic, 
lexical and syntactic systems. Misleading, exaqcierated or reduced 
prosodic cues are possible causes of mi scommuni cation. Prosody is used 
to establish relations across the discourse, as for example , when 
rising intonation at the end of a tone group'serves to hold the 

floor and imolies that more is going to be said, or when stress 
placed on certain words serves to hiqhlight certain information 
or parts~of the discourse. 

5. Provision of Adequate Background Information Prior to 1 
Point Making" ! The provision of background information, i.e., 
who, what, where, when, why and how, in elaborating upon a topic 
or in building a narrative text varied widely between the more 
highly proficient and less profrcient students and among grade- - 
ageJevels.* Younger students and less proficient older students 
tended to relate sequences of activities during elaborated 
discourse without providing specific information about who was 
involved or the circumstances under which the activities took 
place. When questioned by the examiner, these students could 
usually supply the information; however, it wasn't volunteered 
and/or placed at the beginning of an extended discourse segment 
as seen in the data of older or more proficient students. A 
study by Menig-Peterson.and McCabe (1973) indicated that older 
children tended to cluster orientatina or backqround information 
at the beginning>of their narratives and used more backqround 
clauses-T:han did the younger students from whom many of the 
background clauses were elicited directly from. the examiner. 

, It should be hoted that . some of the more . proficient younner * * 
students provided more explicit backqround clauses than the less 
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proficient older students and they sometimes directly questioned 
the examiner to check on the degree of shared knowledge between 
themselves and the listener. A higher percentage of clauses to 
total clauses are spent in providing background information by 
older students. 

One feature -of background contextual i zation or topical ization 
that appears to be related to developmental /acquisitional stage 
was the nonspecific use of pronouns, what Piaqet called the 
overuse of pronouns, in young children's discourse. It was not 
unusual for young children, e.g. Kindergarten and Grade 1, to 
use pronouns in a narrative or description without first 
specifying 'their reference. When this happened it was difficult 
to interpret who the child was referring to without questioning 
him/her or without direct access to the book itself. Nonspecific^ 
reference in narratives from the wordless book gave the stories an 
unplanned and incoherent quality. This occurred both in conver- 
sational discourse where the listener presumably did not share 
the context and in the wordless book where the child may have 
assumed that the adult did share the context as the book itself 
was usually visible to both. King and Rentel (1981) found that 
children's economy and precision of pronoun reference increased 
sharply between Grades 1 and 2. We would expect a greater 
precision in use of reference for older students but in younger 
students overuse of pronouns with nonspecified referents would 
not indicate a lack of proficiency. 

6. Completeness of Infor mation. In elaborating about how 
to make or do something or telling narratives, one indication 
of proficiency is that the topic or narrative is adequately 
described. While there are wide variations in the amount of 
detail any one speaker might include in talking about a subject, 
yet at a very minimum a description or story requires depiction 
of a beginning, middle and end. We provided a model .for 
analyzing the structure of descriptions about how a game is 
played (Chapter IV) and-models for analyzing the episodic 
structures found in the wordless story books used in the study 
(Chapter V). When a child's discourse was somewhat incoherent, 
we found it was sometimes, useful to refer to these models to 
understand how missing pieces of information might be related 
to lack of coherence. Elements related to richness or complexity, 
anci pointmaking discussed below are also aspects of completness 
of information. Moreover, completness of information can be 
conceptualized in various 'ways . The two that were important 
in the. analysis of the data, samples in this study were 
ennumenation or description and summarizing or synthesis of 
information. This is not to deny that all description is 
highly selective of aspects included in discourse. 



7. Richness' and Complexity of Interclausal Relations . 
More proficient students used discourse that was rich in semantic 
complexity. Thi s 'invol ved selection among alternate words 
where choices were readily available and the use of a repertoire 
of various syntactic strategies for relating clauses to show 
complex relations. Originality was also discussed as adding to 
the richness and was observed in some of the student narratives. 
Original means that some aspect of the narrative is unusual, 
unexpected, surprising, in such a way that this original aspect 
can be seen to fit with/or be integrated with other parts of 
the story and thereby appears to be a result of the narrator's 
intentions rather than accidental. The use of the same wordless 
books across students in tie assessment provided an opportunity 
to notice originality in sDme of the narratives. Examples of 
complex semantic relations were cause-effect, event-consequences, 
motive-actions and- event-human response. The integration of a 
variety of -these semantic relations within discourse on a topic 
or in constructing a narrative text was observed in the discourse 
of more highly proficient students. The more proficient students 
also used a variety of different connectors including in 
addition to 'and 1 and 'and then' subordinating conjunctions to 
introduce adverb clauses and make it clear exactly what the 
relation was between clauses. Adverbial clauses were much more 
apparent in the conversational data than the use of adjectival 
clauses although some use of adjectival clauses was rioted among 
older- students. The students use of subordinate adverbial 
clauses is compatible with findings in Loban's (1976) 
language development research but their infrequent use of 
adjectival clauses may indicate a relatively low level of 
language development in this research sample. However, this is 
suggestive only and in need of further research as it is problemat 
to compare findings from two studies using widely different _ 
research methodologies. (Richness was discussed in detail in 
Chapter V, Narrative) . 

. 8. Flexibility and Range of Communicative Competencies , 
More proficient students were able to vary the communicative 
strategies they used to interact in the conversational situation 
and al so, demonstrated command of toore different types of 
strategies, such as making topic shifts explicit, qualifying state 
ments, etc, than less proficienct students. .They were also more 
likely to shift to a narrative style when appropriate, 

9. Pointmaking and Highlighting . More proficient students 
were more likely to be clear about the point they were making in 
their talk and provided more highlighting emphasizing important 
events or factors in their descriptions or narratives. It was ' 
often difficult to discern any point or major emphasis -in the 
discourse of less proficient students and they were less likely 
to make foreground-background relations explicit. In other 
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words, it was more difficult to discern the communicative intent 
of utterances from less proficient students. 

10. Summarizing/Synthesizing . The ability to summarize 
or synthesize information at a more abstract level than that of 
ennumeration or description of sequences of actions was viewed 
as an important proficiency' that aopeared in the discourse of some 
older students. By summarizing, a speaker is able to convey 
information in a relatively refined and condensed fashion and 
may be able to ascertain the needs or interests of the listener 
before going into a more detailed description. Summaries also 
are economical V/ays of establishing a shared context before 
engaging in speech for other purposes, e.g. to give opinions , 
provide an evaluation or make comments. Summarizing may also 
be a more appropriate response when the speaker can assume a 
shared general knowledge background with the listener. 

. 11. Verb Tenses in Narrative Discourse . As stated previously, 
ranges of discourse proficiency are described in terms of multiple 
co-ocurring variables across lexical, semantic v syntactic, and 
prosodic systems. Structure, e.g. tense, can't be evaluated in 
isolation from its meaning in discourse. This was best seen in 
variations in strategies students used to produce a narrative 
text from the wordless books. More highly proficient students 
distinguished foreground from background information by switching 
tenses. In other words, a change in verb tense signalled a change 
in meaning. Less proficient students, while usinq tenses correctly 
in terms of semantic meaning at the clause level, did not use 
verb tenses consistently or flexibly to enhance the meaning of 
their narratives. The inconsistent and seeminqly haphazard use 
of verb tense by the less proficient students seemed unrelated 
to the story line and thus was confusing to the listener. 

Further complicating this issuers the fact that one can't 
tell the function of a clause from its tense. In analyzing 
student narratives from the wordless books it became apparent 
that the function of a clause, e.g. background orientation or 
event sequence, could only be discerned from examining the 
narrative context in which it occurred. This was discussed at 
^length in the chapters on narratives. 



Relating Judgments of Language Proficiency 
to Judgments of Communicative Intent 
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A. The general point needs to be made that judgments of language 
proficiency are based on understanding of co.nmuni cati ve intent . 

Communicative intent involves an understanding of: 

1. The referential world of the discourse . This is. the part 
of meaning closest to information theorists' idea of 
language as a system for communi cati nq "information", 
or Locke's view that it communicates "ideas". However, 
our meaninq of this is essentially different because 
of the importance of the contingency of this referential 
world being built up by the discourse. It is contingent 
first of all on shared assumptions and social practices, 
particularly communicative practices. That means it is 
contingent on the "negotiation" of meanings and inten- 
tions in the interaction. And in addition, creating a 
referential world within which things can be talked about 
"sensibly" or "coherently 11 i s dependent on shared cultural 
ideas. What competent topic talk and/or narrative depends 
on is the use of particular strategies for establishing a 
sharedness within which a story can be constructed or a 
topic discussed. The Strategies themselves only work if 
in. fact they are already familiar to the interlocutors. 
Using a fami 1 iar'strategy makes it possible for a listener 
to know where you're coming from. The elevator example 
of 310B-5-F is a good example of how the child relies on 
knowledge of a widely-known event schema (riding in an 
elevator) and effectively ties her reaction to the schema 
by making that reaction explicit , leaving everything else 
to be inferred or assumed (such as the "cause-effect" - 
relation between elevator speed- and fear). So it is not 
merely a matter of providing background information, but 
of providing just the right amount and kind of information 
in just the right place to establish a shared world. 

2. The point or purpose of the referential world being created 
(if there is one as in topic ta-lk or narrative. Of course 
in greetings there is none). 

When we can follow a child's discourse in every respect in terms 
of understanding communicative intent, we consider him or her 
proficient. The discourse makes sense, is coherent, has point, 

purpose, direction, i.e., is "rational". 

ti , <» 

B. A second-point to be made is: there are varying strategies 
for constructing a sharable discourse world. First of all, . 
these involve multiple levels of "behavior", linguistic 
prosodic, nonverbal.' Within each of these realms there are 
again multiple means: 



1. Linguistic - word choice; syntactic choice; inter- 
clausal relations and cohesive mechanisms (Halliday 
and H-asan, 1976) 

2. . Prosodic - stress and tonal contour; register shift; 

rhythm and tempo shifts. 

3. Nonverbal - facial gesture; body posture and orien- 
tation; hand and arm gesture. 

"No single list of behaviors taken separately will tell one what 
kinds of communicative intent are being brought into play, or 
how they are. One has to look at their integration . But more 
than this , they have to be interpreted in the light of the 
ongoing, developing, contingent interactional context* and in 
"the "larger" sociopolitical, cultural and economic life- 
situations of the. actors. 

Finally, given that there are different forms of discourse in 
the sense that there are different strategies for creating 
coherence; point, etc., some forms may be preferred over others. 
And preferences or expectations will vary for different ages", 
anti for different social roles or persons. 

Therefore , when we judge proficiency it is incumbent on us to • 
make clear what expectations we have in terms of the- forms of 
discourse for given ages or groups. These expectations are in 
fact implied in the ways examiners interacted with the children, 
as well as in their, and our, intuitive judgments of proficiency. 

These are culturally specific expectations grounded in our own 
social practices, which have clearly been influenced by the 
demands of essayist literacy. 

Examiner Training and Scoring Issues 

Examiner training and scoring issues are very large, but we 
believe not insurmountable, obstacles to the application of a 
sociolingtiistic discourse approach to language proficiency 
assessment in school districts. Ideally; a school district 
considering implementing this approach to assessment should be 
committed to an extensive inservice/retraininq program for its 
teachers, based upon current research in sociol i ngui sties , the 
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ethnography of communication and bilingual education. The 
experience of the school district in training bilingual teachers 
in the elicitatiun procedures used in the Language Proficiency 
Measure (LPM) has shown that the teacher-examiner's involvement 
in listening to and analyzing audiotaped data from the 1PM is 
indispensible in training people to conduct valid el i cit- 
ations. In this report, we have discussed ways in which 
elicitors* discourse styles strongly influences' the" complexity 
and coherence of the language produced by children. Character- 
istics of adult interactive discourse that provides an optimal 
context for student performance in a conversational role during., 
an assessment process are similar to that found by Wells (1981) 
to typify the discourse style used by parents of more 
proficient preschool children. In quoting from a number of j 
research studies Wells showed: i 

' What distinguished the speech of the mothers of the 
fast developers was the greater proportion of their 
utterances that were related to the preceding child 
utterances in the form of expansions and extensions 
of the child's contributions. Similar results have 
emerged from the Bristol study, with the additional 
T finding that amount of speech is also, important. 
(Wells, 1981, p. 115) 

"Several issues may be discussed briefly regarding training. • 
First, examiners must be trained to the level where they command 
"clinical" interactive strategies. appropriate to a variety of 
individual student contexts. Second, the discourse skills 
learned in conducting a number of LPM elicitations are quite 
different from the 'known answer 1 examination sequences observed * 
in classroom teacher discourse. Qn the other hand, teacher 
training and sensitivity to individual differences suggest a 
predisposition towards acquiring these skills readily, after 
appropriate training. Furthermore, we suspect that this 
increased sensitivity to context by teachers trained in this 
method would increase their effectiveness as interactants in 
the classroom. Next, the sensitivity to cultural and ethnically 
distinct styles of communication of the linguistic minority 
group being assessed of teacher-examiners from the same ethnic 
background improves the assessment context. Articulation of 
alternatives in communication styles and selection of topics 
among the assessment team members enhances the probability of 
the cultural appropriateness of the assessment procedures. 
Listening to tapes and discussing various interpretations of the 
interactional situation and discourse elicited from students 
as a group is an important part of tra-ining examiners. 
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Scoring Issues , There is a vast difference between applying 
socio! inguistic-discourse analysis methods directly to samples of 
audiotaped and transcribed discourse as described in this report 
and developing scoring methods that can be used in school district 
assessment by nonresearchers. The district discourse analysis 
committee has been developing methods which are both an application 
and a modification of the Framework for Analysis to the analysis 
and scoring of student discourse from the LPM for a year and a 
hal f . As described in the endnote to Chapter II, Methods, the 
first attempt involved a checksheet while the second attempt 
resulted in a scale of three discourse variables each for both 
tasks, the conversation on global topics, and the narrative from 
the wordless book. Presently under. development is a scoring 
system intended to provide a profile of student's proficiencies 
for each task and language. The profile is intended to incor- 
porate the main findings of this research^ in as much as is 
practical in applied settinos. 

The following issues- or questions need 'to be addressed in 
applying these methods to large scale assessment in. school 
districts . 

1. "Practicality." t How much time can be spent in scoring 
one tape? Is it feasible to use a scoring technioue requiring 
multiple replaying of tapes? How many discourse features can 
be coded in the scorinq process while still restricting the 
amount of time spent on scoring any one tane? Is it Dossible 
for the same person, i.e., the examiner, to both administer and 
score the LPM either at the same time or at different times? 

If not, how is information not on the tape, e.q. nonverbal 
behavior, communication previous to taping, etc., going to be 
considered in the proficiency judgment? How much time would it 
take to train persons to ,do the analysis? What would the dollar, 
cost and educational benefits be for a school district to 
conduct this kind .of assessment? 

2. "Holistic vs. Specific Indicators of Proficiency . " We 
have recommended that a primary focus in the analysis of 
discourse is upon coherence or comprehensibil i ty of longer 
discourse units called topics or narratives. Yet scoring 
involves a reduction of these wholes into something more 
specific which may result in the person doing the scoring losing 
track of meaning being communicated. In making judgments about 
proficiency it is important to consider the communicative 
strengths and weaknesses of the entire elicitation before 
examining specific features so that these can be evaluated 
within context. Coding of specific features is important, 
however, in constructing languaqe Drofiles for feedback to 
teachers and curriculum Dlanners\ Lanquaqe acquisition and 
development is too complex to be adquately described by labels 
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such as limited, functional and proficient as would be derived 
from holistic scoring and grading of entire discourse samoles. 

3. "Differentiating Between Types of Scores." In scoring 
a discourse sample we have to recognize that some indicators of 
proficiency that might be scored are one-directional in that 
students who perform consistently well might be inferred to 
have mastery of that competency but that those who did not 
cannot be assumed to lack the skill. This would be especially 
true of conversational management strategies and topic elabor-^ 
ation. (Various types of scoring derived from competency testing 
are discussed by Messick, 1981.) There also appears to be a need 
for scores in some areas, use of complex syntax, that would give 
the student credit for premastery stages of acquisition. Further, 
as the performance of students regarding certain aspects of 
discourse may be inconsistent and even students generally low in 
overall proficiency may display a competency once there is a 
need to depict frequency or how the student characteristically 
performs . 

4. "Reliability of Scoring." It is necessary that any 
aspect of discourse upon which proficiency is evaluated receive 
the same score from multiple evaluators. Holistic scoring of 
writing samples has been shown to be reliably. In selecting those 
discourse variables to be included for scoring in assessment only 
thosefor which high reliability can be established will be included 
It would appear that there is a higher probability that certain 
discourse or interactional competencies that appear on tapes 
would be more easily discernable and hense reliably scored than _ 
others. For instance, deciding on such features as whether a 
student has elaborated on topics,- provided background information, 
negotiated a misunderstanding and/or developed a narrative 
containing complex semantic relations seems an easier and hence 
more promising candidates for a reliable scoring system than 
does decisions regarding originality in narratives. Another 
problem in establishing reliability will occur in, marking 
continuous scores, e.g. "Characteristically,' 1 "Sometimes," 
"Seldom" or "Never" generalizing about the whole discourse 
sample for a task. An important question for consideration by 
policymakers is:"Are the more easily scored discourse elements 
in terms of time and reliability, e.g. certain syntactical _ 
features, as important in terms of socio! ingui stic sensitivity 
to the language competencies of linguistic minority students 
as other more complex aspects of discourse, e.g. varying verb ^ 
tense for highlighting foreground from- background information?"' 
For those who feel the sociol ingui stic approach is the best 
alternative for language proficiency assessment the bottom line 
will be whether a reliable and practical scoring method can 
be developed which incorporates enough features of discourse 
competencies to be a viable measure of the aporoach and whether 
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evaluations of elicited discourse generalize to students' communi- 
cative competencies observed in other relevant contexts. 

Advantages and Disadvantages 

In summary, tine advantage of this approach to language 
proficiency assessment is that it measures language in one context 
in which it is used. It is an approach that can be used with 
students at various stages of language acquisition and attempts 
to provide an optimal getting for linguistic minority students 
to display optimal performance. Skills learned by teachers 
regarding the elicitation and analysis of discourse are poten- 
tially relevant to adult interaction with students in other 
contexts such as the classroom. 

Disadvantages of the approach are that much research remains 
to be done because it is a new type of mea:urement , examiners and 
discourse analysts must be thoroughly trained to a level of 
'clinical 1 skills, methods of scoring and providinq feedback to 
classroom teachers need to be established and the method is more 
time consuming than other testing approaches. 

Need for Future Research ■ 

The need for further research was implied in the limitations 
and other sections of this report. Directions for future 
research and expansions of this present study are suggested 
below: 

1. There is a need to compare evaluations of language 
proficiency based on elicited discourse samples to 
discourse observed in more naturalistic settings. 
For instance, jtea'cher ratings and ^observations of 
student ^discourse could be compared with LPM results. 
Naturally occuring narratives could be compared with 
elicited narratives. Elaboration during play or in 
classroom discourse could be compared to that in the 
el ici tations . 

2. There is a need to explore the relationship between 
achievement and language proficiency in bilingual 
children. „ ' 

3. There is a need to explore more fully the range of 
proficiency in both Spanish and English. We would 
especially recommend that the range of proficiencies 
of a larger qroup of students, e.g. 25, from limited 
to proficient in English, all of whom are proficient 
in Spanish be studied, holding age and qrade level 
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constant, to better understand the acquisition of 
English for children coming'to school monolingual 
in Spanish or bil ingual . 

In our own data, further research could be done comparing 
proficiencies across tasks and languages for individual 
bilingual students to more fully specify similarities 
or differences in their acquisition of discourse 
strategies in Spanish and English. 

We have not discussed in any detail how the material in 
this report is to be used in practice 1n making decisions 
about placement and for developing instructional programs 
for linguistic minority students. We fe.el that this 
approach to conceptualizing language proficiency under 
the broader construct of communicative competency holds 
promise in the development of more successful programs. 
We plan to include recomniendations about this in future 
work some of which, however, require verification 
through ethnographic observations outside of the 
assessment context. Briefly, we feel that overall, 
the language assessment procedures undertaken by the 
school district and the- research team, involving 
relatively large numbers of teachers, researchers and 
. teacher-researcher collaborators in the development of 
more appropriate language proficiency measures was 
valuable in developing greater awareness about what is 
involved in being communicatively competent, and also 
about contexts that promote or retard the linguistic 
minority chiTcT's language development. .In traditional 
language proficiency approaches , contextual variables are 
hidden from view and consequently ignored. Therefore an 
extremely important variable .influencing language 
performance, that of context, is ignored not only in_ 
judgments of language proficiency but also in educational 
^■programs designed to meet the needs of linguistic 
minority students. An important aspect of a socio- 
linguistic discourse approach to language proficiency 
assessment is its insistance on paying attention to 
context. We have found in our work with teachers and 
in presentations at educational conferences for 
nonspecial ists that there is little awareness of socio- 
linguistics and discourse ' analysis in the field of 
education. We feel this approach is a potentially 
valuable and, as yet, untapped source of knowledge 
for teacher training and development. Further, we feel 
that future rese^rc^ wi 1 1 i ndicate a clear relationship 
between the development of more autonomous oral 
proficiencies and literacy, a relationship that has 
recently been seen in the work of King and Rentel (1981) 
and Sulzby (1981). 
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